Information-based Complex- Ity and Information-based Optimization Information-based Complexity Computational Complexity of Optimization Problems Information-based Optimization Model of Computation Real Number Model Turing Machine Model Linear Programming

This article concerns optimization in two senses. The rst is that information-based complexity (IBC) is the study of the minimal computational resources to solve continuous mathematical problems. (Other types of mathematical problems are also studied; the problems studied by IBC will be characterized later.) J.F. Traub and A.G. Werschulz 14] provide an exposi-tory introduction to the theory and applications of IBC, with over 400 recent papers and books. A general formulation with proofs can be found in J. The second is that the computational complexity of optimization problems is one of the areas studied in IBC. S.A. Vavasis 16, pg. 135] calls this information-based optimization. We will discuss information-based complexity and information-based optimization in turn. Information-based complexity. To introduce computational complexity, we rst deene the model of computation. The model of computation states which operations are permitted and how much they cost. The model of computation is based on two assumptions: 1. We can perform arithmetic operations and comparisons on real numbers at unit cost. 2. We can perform an information operation at cost c. Usually, c 1. We comment on these assumptions. The real number model (Assumption 1) is used as an abstraction of the oating-point model typically used in scientiic computation. Except for the possible eeect of roundoo errors and numerical stability, complexity results will be the same in these two models. The real number model should be contrasted with the Turing Machine model, typically used for discrete problems. The cost of an operation in a Turing Machine model depends on the size of the operands, which is not a good assumption for oating point numbers. For a full discussion of the pros and cons of the Turing Machine and real number models see 14, Chapter 8]. Whether the real number or Turing Machine model is used can make an enormous diierence. For example, L.G. Khachiyan 3] shows that linear programming is polynomial in the Tur-ing Machine model. In 1982, J.F. Traub and H. Woo zniakowski 15] showed that Khachiyan's algorithm is not polynomial in the real number model and conjectured that linear programming is not polynomial in this model. This conjecture is still open. The purpose of information operations (Assumption 2) is to replace the input by a nite set of numbers. For integration, the information operations are typically function evaluations. Computational complexity of high-dimensional integration. We illustrate some of the important ideas of IBC with the …